Prior interpretive execution was obtained in a CEC having multiple independent OSs to increase the CEC's execution efficiency, which is highest when each OS in the CEC is able to have its CPU instructions executed interpretively. Having multiple independent OSs in a CEC requires that the CEC resources be apportioned among the OSs, and that the instructions executed under each OS be restricted to using only the CEC resources assigned to its OS--for example, to execute only within the memory range assigned to the respective OS in the main storage of the CEC.
The execution of multiple OSs in a CEC requires either software or hardware enforcement of resource restrictions and limitations provided for the respective OSs. Enforcement may require the intervention of a higher level operating system for the operating systems--this higher level operating system is called a hypervisor. A software hypervisor provides software enforcement of the OS resource restrictions, which intervenes to provide software limit checking for each instruction executed by any OS. This software hypervisor checking exacts a heavy system overhead burdon on the OS execution, since the OS execution must be interrupted for each instruction execution--this hypervisor interruption is herein called "interception". The software hypervisor interceptions greatly slow the instruction execution process compared to the execution of the same instructions in a similar CEC operating with only a single OS (native mode) which does not require limit checking.
Recent IBM CECs have provided hardware and/or microcode limit checking for instructions executing in multiple independent OS environments. This hardware/microcode limit checking has the advantage of eliminating the hypervisor interceptions as long as the execution stays within the prescribed restrictions and limits. The use of hardware/microcode checking to eliminate interception is herein called "Interpretive execution of an instruction". "Interpretive execution" can execute an instruction at nearly the same speed as execution in a similar CEC operating with only a single OS (native mode), which does not require checking.
With interpretive execution, hypervisor interception is only required if the instruction execution violates the prescribed limits and other restrictions, which should rarely occur.
A technique for obtaining interpretive execution was to use the S/370 "start interpretive execution" (SIE) instruction. Execution of the SIE instruction invokes hardware/microcode checking in a CEC.
Each SIE instruction locates an operand called a "state description" (SD), which is a control block in a CEC's main storage which defines a particular set of resource limits and restrictions. The SIE instruction execution is architected to check the execution of other instructions for violations of the resource restrictions and limits defined in the SIE instruction's SD. The instructions being checked are in a predefined set of IBM-architected CPU instructions.
This predefined set of architected CPU instructions for executing a program with the resource constraints in the SD of a particular SIE instruction is referred to as a "virtual CPU". Any OS may have one or more virtual CPUs running under it.
The resources defined in the SD for any virtual CPU are restricted to the resource boundaries defined for the OS under which that virtual CPU is to execute. Any number of virtual CPUs may execute under each of the multiple OSs in a CEC. The hypervisor predefines the content of any number of SDs to provide any number of virtual CPUs in a CEC. The hypervisor selects and executes one of its SIE instructions having a predefined SD to dispatch a virtual CPU for any OS.
The SIE instruction is executed by the hypervisor when it dispatches any OS to interpretively-execute instructions. The SIE instruction execution continues while the OS instructions are being interpretively executed.
In the prior art, the SIE instruction has been applied to I/O resources of a CEC, including in U.S. patent application Ser. No. 4,843,541 (PO9-87-002) and application Ser. No. 07/752,149 (PO9-91-035), now U.S. Pat. No. 5,222,215. Those inventions obtain interpretive execution for commonly-used CPU instructions, such as the "start subchannel" instruction.
Interpretive execution has been disclosed in the prior art, such as in the publication entitled IBM System/370 Extended Architecture Interpretive Execution (form number SA22-7095-1) which discloses the SIE instruction and its state description operand. The CPU and I/O architecture are dislosed in the IBM Enterprise Systems Architecture S/390 Principles of Operation (form number SA22-7201-00). Also, I/O architecture is disclosed in the IBM Enterprise Systems Architecture S/390 Common I/O Device Commands (form number SA22-7204-00). These describe the I/O architected structure used in the IBM S/390 systems, such as the IBM S/9000 Central Electronic Complexes (CECs). The I/O ESCON channel architecture is disclosed in the ESA/390 "ESCON I/O Interface" (form number SA22-7202-01).
A Channel Subsystem Call (CHSC) instruction has been used in prior IBM systems; it is an instruction with a single operand which addresses a "command request block" which contains an operation code field that is capable of representing a very large number of operation codes, each of which designates a special command function for the CHSC instruction. These I/O channel subsystem commands perform widely-varying types of I/O functions. Some of the prior CHSC instructions operated synchronously. Other prior CHSC instructions operated asynchronously. Asynchronous CHSC commands performing dynamic I/O reconfiguration functions are disclosed in previously cited U.S. Pat. No. 5,170,472 by R. Cwiakala et al which discloses and claims: a "change channel path configuration command", a "change control unit configuration command", and a "change I/O device configuration command". These asynchronous CHSC commands dynamically change the I/O configuration of a CEC while the CEC is normally operating.
Shared I/O resource concept uses the image identifier (IID) invention described and claimed in application Ser. No. 07/898,867 (PO9-92-016). The IID invention is generally independent of whether or not IIDs, themselves, are hidden from the operating systems (OSs) in the CEC. That application expresses a preference for the IIDs being hidden from the OSs to enable old versions of OSs to operate within a CEC modified with the IID invention to allow shared I/O resources among the OSs. Then, old OS versions may be run in a CEC also running new versions of OSs in its different partitions.
Multiple OSs in a CEC are coordinated by a hypervisor, in which the CPU and storage resources of the CEC are divided among the independently executing OSs. Some hypervisors are structured in internal code (e.g. microcoded hypervisor) and other hypervisors are structured in software (e.g. software hypervisor). A commercial example of a microcoded hypervisor is the IBM S/390 PR/SM (processor resource/system manager), which co-ordinates resource contentions among independently executing OSs in separate logical resource partitions (LPARs) of a CEC such as the IBM S/9000 model 900. An example of a commercial software hypervisor is the IBM S/370 VM/MPG (virtual machine/multiple preferred guests) system running on an S/370 3090 model J, in which so-called virtual machines (called preferred guests) execute separate OSs in respective logical resource partitions divided by the system software in a software directory. A preferred guest (sometimes called a V=R or V=F guest) operates with only one level of address translation, while a non-preferred guest (sometimes called a V=V guest) operates with two levels of address translation (which has signifantly higher system overhead than one level of translation).
In such prior multiple-OS systems, an I/O channel and its associated subchannels can only be directly used by a single OS without hypervisor intervention (pass-through operation). Hypervisor intervention requires an OS to interrupt its operation while the hypervisor accesses requested I/O channel or subchannel on behalf of the OS. This intervention uses the system inefficiently because it involves CPU execution of a large number of hypervisor instructions that are eliminated with pass-through operation, which eliminates the need for hypervisor intervention.
The one or more central processors (CPUs) in a CEC access a shared electronic storage (MS) in the CEC. The MS of a CEC is divided up among the CEC partitions containing the OSs, and each OS only uses its own part of MS to execute its programs and is not permitted to access any part of MS partitioned to any other OS.
The various resources used by the CPUs in a CEC are divided among the OSs by using a plurality of directories or state descriptors (SDs) in system memory. A "virtual CPU" is represented by each SD or by each section of a virtual machine (VM) directory. The OSs execute on the virtual CPUs within their respective logical-resource partitions of a CEC.
A hypervisor controls the overall operation of the CEC, including the dispatching of OSs on the central processors (CPUs) in the CEC, and the resolving of conflicts among the OSs.
A software hypervisor dispatches an OS by invoking software accessing the section of its directory associated with the virtual CPU being dispatched. Each VM directory section is associated with a particular OS through the subset of system resources it assigns to the virtual CPU.
A microcoded hypervisor dispatches an OS by executing the SIE instruction representing the virtual CPU in the OS being dispatched. Each SIE instruction is associated with a particular OS through the subset of system resources assigned to the virtual CPU in its SIE SD.
Each OS controls the dispatching of all supervisor and application programs which run under the respective OS without hypervisor involvement, unless an exception occurs.
Early hypervisor systems required the hypervisor to control all I/O operations for all OSs in the system (e.g. early VM/370 software) including having the hypervisor control all channel operations, start all subchannels for all I/O devices on a channel, and handle all I/O interruptions from its devices for all programs running under all OSs. Any OS running as a "guest" under the hypervisor had to be interrupted to allow the hypervisor to handle the I/O operations requested by the guest.
A hypervisor is used in U.S. Pat. No. 4,843,541 (PO9-87-002) entitled "Logical Resource Partitioning of a Data Processing System", assigned to the same assignee as the subject application, which describes and claims a system having "I/O passthru" to enable each OS in a CEC to handle its own I/O operations using dedicated I/O channels and devices without involving the hypervisor. This passthru (or passthrough) feature allowed each OS to start I/O operations requested by supervisor and application programs running under an OS, and allowed the OS to handle the I/O interruptions resulting from such I/O start operations. The hypervisor was only needed to intercept an OS channel operation when an exception condition occurred. That invention is used in the IBM PR/SM LPAR and S/370 VM MPG systems.
U.S. Pat. No. 5,222,215 Ser. No. 07/752,149 (PO9-91-035) filed on Aug. 29, 1991, entitled "CPU Expansive Gradation of I/O Interruption Subclass Recognition", assigned to the same assignee as the subject specification, enables a significant increase in the number of logical partitions (LPARs) and CPUs runnable in a CEC. This application enables each of the CPUs in a CEC (executing any OS running in the CEC) to handle all of the I/O interruption subclasses available in the system. This avoided a prior pass-thru constraint that restricted each OS to only handling interruptions for one of the I/O interruption subclasses available in the system.
A CEC hypervisor may or may not be assigned its own resource partition. A microcoded hypervisor may not need a partition, but a software hypervisor needs a partition to contain its software.
For security reasons, the system resources are specified differently for the hypervisor than for the OSs in the CEC. Only the hypervisor views each SIE instruction's operand which is a state description (SD) control block that defines system resources for a virtual CPU in the CEC. Each OS has other control blocks defining resources available to the OS. The I/O subsystem of a CEC is used by all OSs, although many of the I/O resources controlled by the I/O subsystem may be dedicated to only a single one of the multiple OSs in the CEC.
Prior S/390 CECs have I/O subsystem which contains one or more I/O processors (IOPs) and an I/O channel processor for each channel. The channel connects to one or more I/O control units, and the control units connect to I/O devices. Each IOP communicates with the CPUs and the channel processors in the CEC. Each channel processor is dedicated to controlling an associated channel, which use may transmit data in bit-serial or bit-parallel form, or with a combination of bit-serial and bit-parallel data transmission. (A currently bit-serial type of channel uses fiber optics with the IBM ESCON architecture.) The terms "channel" and "channel path" mean the same thing in this specification.
The asynchronously operating IOPs pass CPU requested I/O work to the channels, and may further include channel processors that receive the work requests from the IOPs to directly control the operation of the respective channels. Each channel processor may be dedicated to a particular OS, so that it can continuously sense the busy state of its channel.
A subchannel is specified for each I/O device supported by an OS under the IBM S/390 architecture. A SCHIB (subchannel information control block) is an OS control block used by an OS to specify resources usable by a subchannel, including the subset of channels usable by the subchannel. Each SCHIB contains fields for a up to eight channel identifiers, called channel path identifiers (CHPIDs), each of which specifies a channel which may be selected for use by the subchannel. An available one of the specified CHPIDs is selected for each data transmission request of the subchannel which is not busy at the time of a CPU subchannel request. Thus in prior CECs, only the channels assigned to the OS could be selected for any of its subchannels, since prior systems only assigned a channel to one OS which could not share channels.
Each prior I/O subsystem has a control block for each I/O resource which is only used by the CEC microcode, and the I/O subsystem control blocks cannot be accessed by either the hypervisor or OSs. However, special instructions are provided to enable predetermined I/O subsystem functions to be interfaced by both the hypervisor and the OSs when they execute on any CPU in the CEC.
A large number of subchannels are configurable in a CEC (up to 64K), but only a small number of channels are configurable in a CEC (up to 256) in the S/390 architecture.
Configuring a CEC's I/O subsystem involves more than merely considering physically-connected I/O resources, such as a CEC's physical-connected channels, physical-connected control units, and physically-connected devices. Control blocks are provided in the I/O subsystem microcode to electronically control these physical resources. Without these control blocks, the I/O resources alone would be useless to the CEC, because the physical entities alone of these I/O resources cannot be controlled to do work for the CEC.
Accordingly, merely connecting a physical resource to a CEC does not configure that resource into the CEC. This is because the existence of the resource cannot be recognized by the CEC hardware and software until the resource is represented by a control block, which is the entity looked to by the CEC hardware and software for controlling the physical resource.
In the prior art, a CEC's I/O configuration requires having one control block for each physically-connected I/O resource. The control block contains all of the information needed to control the corresponding I/O physical entity; for example, a busy bit in each control block is provided to indicate whether the physical entity is busy or not, as well as all kinds of other control fields for indicating the current physical characteristics of the entity.
In prior S/370 and S/390 architected computer systems, each channel was configured in the CEC by constructing a single channel control block (CHCB) in the CEC's I/O channel system storage. Each I/O device was configured in the CEC by constructing a single subchannel control block (SCB) in the CEC's I/O subsystem storage. One or more logical control units were configured in the CEC as a shared device cluster by constructing a single logical control unit control block (LCUCB). A shared device cluster is defined as a set of one or more logical control units that share devices.
The CHCBs, SCBs and LCUCBs are structured in the I/O subsystem storage, which is not accessible to the OSs, but is accessible to the CEC's hardware and internal code (microcode) to electronically control the corresponding I/O resources physically connected to the CEC.
A control block configured in the system may indicate several states for its physical I/O entity, such as if the entity is usable or not by an OS. For example, the physical resource may be subjected to a "vary off line" command which sets a field in its control block indicating the physical resource is not available for use. Or a "vary on line" command sets the control block field to indicate the physical resource is available for use, although it may be currently busy (indicated by the busy field) and therefore not currently selectable.
Prior art reconfiguration of the CEC's I/O subsystem involves changing these I/O subsystem control blocks. Reconfiguration may add a new control block into the I/O subsystem storage to bring a corresponding I/O resource into the CEC configuration, whether the physical resource was previously-connected or is newly-connected. Or, reconfiguration may delete an existing control block from the I/O subsystem storage to eliminate the corresponding I/O resource from the CEC configuration, even though the resource is still physically connected to the CEC.
Furthermore, two fundamentally different methods are provided in the prior art for obtaining reconfiguration of the CEC's I/O subsystem. They may be generically referred to as static and dynamic reconfiguration methods.
An adverse consequence was that when a dedicated channel or device was utilize only a small percentage of time by its assigned OS, the channel or subchannel could not be dynamically switched to another OS using passthru; only non-passthru hypervisor accessing was available with its resulting inefficiencies. Consequently, dedicated channels generally remained under-utilized. (The available manual switching of a channel to a different OS did not permit dynamic online switching of an I/O channel to another OS.)
Limiting the number of channels to each OS had the effect of limiting the I/O data rate available to the OS by restricting the number of simultaneous parallel paths for data transmission.
A fundamental change in the control block structure of the I/O subsystem of the CECs is made by the invention in application Ser. No. 07/898,867 (PO9-92-016), which provides the concept of "sharing sets" of control blocks for enabling the sharing of I/O resources by plural operating systems (OSs) in a CEC, including I/O channels, and control units and devices using shared channels. The sharing set concept replicated the control blocks for the same resource and distinguished the control blocks in a sharing set by assigning them different image identifiers (IIDs). The different OSs occupy different logical partitions of the CEC. These partitions are considered logical partitions because they use different parts of the same CEC physical resources, or time share the same resources in the CEC; for example, the main storage is divided by microcode defined boundaries which can easily be changed without changing the physical structure of the main storage.
The operation of this invention is dependent on the use of the invention described and claimed in U.S. patent application Ser. No. 07/898,867 (PO9-92-016), which describes a novel I/O resource control method and means that enables an I/O resource (such as an I/O channel, subchannel (device) and logical control unit) to be dynamically shared (or operated unshared) by a plurality of operating systems (OSs) executing in a CEC.
Application Ser. No. 07/898,867 (PO9-92-016) describes a method for increasing the connectivity of I/O channels to a multiplicity of operating systems (OSs) running in different resource partitions of a central electronic complex (CEC) to obtain sharing of the channels among the OSs in a CEC. That application discloses the use of image identifiers (IIDs) for assigning I/O resources to different OSs. In the channel subsystem of a CEC, each shared I/O resource is provided a sharing set of control blocks (CBs) in which a respective CB is assigned to (and located by) a respective IID of one of the OSs running in the CEC. Each of the CBs in a sharing set provides a different image to each OS of the same I/O resource. The different CB images are independently set to different states by I/O operations with the resource for the different OSs, so that the OSs can independently operate the same I/O resource without regard to the order of use of the resource by the OSs. The IID used in executing an I/O request by an OS is transmitted to the I/O control unit used to access a requested I/O device and is stored by the control unit as part of the logical path connecting to the control unit for later use by the control unit in responding back to the requesting OS for the I/O request.